This invention relates to screening persons for security threats through the automated analysis of body scanner images.
A variety of body scanners have been developed to detect weapons, explosives and contraband concealed under the clothing of persons entering security controlled areas. These devices operate by detecting radiant energy that has been modulated by or emitted from the body of the person being examined. Radiant energies used include: x-rays, microwaves, millimeter waves, infrared light, terahertz waves, and ultrasound. In a typical operation, the person being examined is actively exposed to a beam of millimeter waves or x-rays. A portion of this radiant energy interacts with the person, their clothing, and any concealed objects they may be carrying. This interaction modulates the radiant energy reflected from, scattered by, or transmitted through the person. This reflected, scattered or transmitted radiant energy is collected by sensitive detectors, and the resulting electronic signals are routed to a digital computer. Alternatively, some body scanners operate passively, collecting radiant energy that has been thermally emitted from the person's body and surrounding area. Examples of this are infrared and millimeter wave sensitive cameras. Regardless of active or passive operation, body scanners convert the electronic signals from their detectors into digitally represented images of the person's body. In these images the clothing is essentially transparent, allowing the security officer to visualize objects that are concealed underneath the clothing. Commercial body scanners include the model AIT84, sold by Tek84 Engineering Group, San Diego, Calif.; model SECURE 1000, sold by Rapiscan Security Products, Torrance, Calif.; model SmartCheck, sold by American Science and Engineering, Billerica, Mass.; model ProVision, sold by L-3 Communications, Woburn, Mass.; and model Eqo, sold by Smiths Detection, Edgewood, Md.
FIG. 1 shows examples of the wide variety of body scanner configurations. In one configuration 100, the person being screened 12 stands motionless within the apparatus 101 performing the scan. In another configuration 102, the person 12 stands next to the scanning device 103. In yet another configuration 104, the person 12 walks along a path 20 through an archway body scanner 105. Many other configurations are possible, including: cameras detecting infrared or microwave energy, the person being screened turning their body in front of a scanning panel, and standoff systems where the person is scanned from a considerable distance from the apparatus. Examples of body scanner images are shown in FIG. 2, created from backscatter x-rays 30, transmitted x-rays 31, and passive microwaves 32.
In spite of using different radiant energies and imaging geometries, body scanners detect concealed objects in the same fundamental way: they create an electronic image of the person with the clothing being essentially transparent. This electronic image is composed of bits of digital information, which may reside in a storage medium, computer processing unit, or other device capable of retaining such data. For image storage this may be done in a common file format, such as jpg, bmp, or gif. Within a computer processing unit the storage will typically be pixel values ordered in a row and column arrangement. The electronic image can be manipulated directly in digital form, or converted to a visual image by devices such as image printers and video monitors. As used here, and commonly in the art, the term “image” and “scan” refer to the bits of digital information residing in a digital device, the visual display of this information on a video monitor, the printed image corresponding to this digital information, and other such data presentations. These concepts of digitally representing and manipulating images are well known in the art of image processing.
All body scanners incorporate a digital computer, as shown in FIG. 1 18 that receives or otherwise acquires an electronic image from the imaging assemblies. In the most basic operation, this electronic image is displayed on a monitor 16, either mounted directly on the scanning apparatus or located in close proximity. The security officer 14 evaluates the displayed image through his innate and trained ability to recognize the normal visual appearance of the human body. That is, the security officer knows what the human body looks like and can therefore detect objects appearing in the displayed image that do not correspond to human anatomy. In addition, from his training and experience in society, the security officer can often recognize which of the concealed objects are benign and need no investigation, such as wallets, watches, coins, buttons on the clothing, and so on. If the security officer observes an object that is not a part of the subject's body, and is not recognized as benign, the security officer confronts the subject to determine if the object is a prohibited item. This may be as simple as asking the subject to remove the item from within their clothing, or as invasive as a strip search. The method of resolution depending on the characteristics of the object observed and the policies of the security facility being entered.
Body scanners are capable of detecting a wide range of security threats and contraband; however, the required image interpretation by the Security Officer presents a multitude of problems and difficulties. The manpower requirement to operate these systems is very high, requiring a security officer to be present to analyze each image. This is aggravated by the specialized training each security officer must go through to be proficient in the image analysis. The inherent limitations of human image analysis, and the nature of this work, promotes errors and deficiencies in the security screening process. For instance, security officers may become distracted or tired and miss concealed objects. In a worse scenario, a security officer may be bribed or coerced to ignore concealed objects. Further, human image analysis requires that the examination area be large enough to accommodate the extra security officer. Further, humans require about ten seconds to process each image, which can slow the throughput of the security checkpoint. Still further, some persons object to an electronic image of their unclothed body being displayed and viewed by the security officer.
FIG. 3 illustrates a configuration directed at overcoming these problems of operator image interpretation, commonly know as “Automated Target Recognition,” or ATR. In this approach the human operator is replaced by a digital computer running specialized software 90. The electronic image 70 produced by the body scanner contains a digital representation of the person's body 71, as well as any concealed objects 72, 73. This digital information 74 is passed into the digital computer 90. This may be the computer operating the body scanner, as shown in FIG. 1 18, or a separate device connected to the apparatus through a communication network. The goal of ATR software is to discriminate between features in the image that correspond to the person's anatomy 71, and features that correspond to concealed objects 72, 73. The result of this operation is digital data 84 representing only the concealed objects, and not the human body. The Security Officer operating the body scanner is then notified of this information 84 through an annunciator in some convenient way. Most commonly, this is done through a graphical representation 80 displayed on the system's monitor 18. A cartoon-like outline 81 may be used to provide positional reference. However, this outline 81 is the same for all persons being scanned, and does not correspond to the anatomy 71 of the particular human body being examined. Also most commonly, concealed objects 72 73 in the electronic image 70 are displayed to the operator as boxes 82 83, respectively, or some other pattern in the graphical display 80. An additional problem in ATR is the presence of common benign objects that cannot easily be divested from the person being screened. This includes jewelry, zippers, buttons and the like. It is highly desirable or necessary to allow these objects to remain on the person during the body scan.
The performance of ATR can be statistically measured in terms of the probability of detecting certain types of concealed objects, versus the false alarm rate. A well performing system detects a high percentage of the concealed objects with minimal false alarms. Conversely, a poorly performing system has a low detection probability, and a high false alarm rate. Humans perform exceedingly well at this task with a seemingly trivial effort. This can be appreciated simply by looking at the high-quality body scan image 70 in FIG. 3. The human brain can immediately separate the person's body from the concealed objects. In stark comparison, prior art ATR has surprisingly poor performance at this task. While the numbers are arguable, in can generally be said that the capability of prior art ATR is orders-of-magnitude below that of human image interpretation. The reasons for this have not been known; it has been a longstanding and frustrating mystery to the scientists and engineers working in the field.
Prior Art approaches generally employ one of two strategies for ATR. The first uses reference information that is contained within the image being examined. One such scenario is described in U.S. Pat. No. 8,194,822, to Rothschild et al., issued Sep. 28, 2010. In this '822 Invention, the value of a particular pixel is compared against a plurality of other pixels values in the same image, providing a reference to detect uncommonly bright or dark areas. Another Invention using this first general strategy is described in U.S. patent application Ser. No. 62/406,702, to Smith, filed Oct. 11, 2016, and incorporated herein by reference. In this Invention, an image feature at one location in the image is compared to the bilateral location in the same image, thereby detecting features that are bilaterally symmetric. In general, human anatomy is bilaterally symmetric, while concealed threats are bilaterally asymmetric. Therefore, this provides a method for separating anatomic from threat features, by only using information contained within the single scanned image.
The second general approach to ATR is to use reference information in a library, i.e., a database, of previously acquired scans. This strategy is described in U.S. Pat. No. 5,181,234, to Smith, issued May 22, 1991, and incorporated herein by reference. In the '234 Invention, a plurality of body scanner images is acquired and analyzed for the presence of image features. These features are stored in a library for later comparison to scans of actual subjects. Features appearing in the actual scans, which do not appear in the library, are classified as potential threats. As briefly stated in the '234 patent: “The location of detected features can be referenced to the absolute location in the image, or in relation to the body of the person being examined” (col. 14 lines 41-43). However, '234 is silent on how a location “in relation to the body” can be calculated or otherwise determined.
Inadequate ATR has placed severe limitations on the use of body scanners. Security personnel at airports, military bases and Government facilities have been faced with undesirable alternatives. One alternative is to use body scanners with human image analysts, providing excellent detection capability and few false alarms. However, they also must accept the associated manpower problems, long analysis times, and privacy concerns. The other alternative has been to use body scanners with prior art ATR. This provides high-throughput, reduced personnel requirements and far better privacy to the person being screened. However, in this alternative, the primary purpose of the body scanner is largely defeated, a result of the poor detection probability and frequency false alarms. A third alternative, which is often selected, is to not use body scanners because of the unacceptable problems of either using, and not using, prior art ATR. Indeed, the performance of ATR is the critical factor in the widespread use of body scanners in security facilities.